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PREFATORY  NOTE 


This  paper  is  based  on  a presentation  given  at  the 
19th  Conference  of  the  Military  Testing  Association, 
October  17-21,  1977,  at  San  Antonio,  Texas.  The  Con- 
ference was  hosted  by  the  Air  Force  Human  Resources 
Laboratory  and  the  Air  Force  Occupational  Measure- 
ment Center. 

Dr.  Robert  Vineberg  and  Dr.  Elaine  N.  Taylor  are 
Senior  Staff  Scientists  in  the  Carmel  Research  Office  of 
HumRRO’s  Western  Division.  The  information  pre- 
sented in  this  paper  was  developed  during  the  course  of 
research  in  Project  SINBAD,  “A  Proposed  Study  of  the 
Job  Proficiency  of  Men  in  Different  Mental  Categories: 

I.  Development  of  an  Instrument  for  Generating  Pro- 
ficiency Profiles.” 


WORKER  ORIENTED  AND  JOB-ORIENTED  INSTRUMENTS 
FOR  EVALUATING  JOB  PERFORMANCE1 . 2 


Robert  Vineberg  and  Elaine  N.  Taylor 


l am  going  to  describe  two  instruments  for  obtaining  performance  ratings.  These  two 
instruments  have  been  developed  at  the  task  and  element  level  of  jobs.  These  instruments, 
the  Performance  Analysis  Inventory  (PAI)  and  the  Task  Proficiency  Inventory  (TPI)  were 
developed  as  part  of  an  ONR  study  concerned  with  the  performance  capabilities  of  men 
at  different  aptitude  levels.  The  part  of  the  study  concerning  aptitude  levels  has  not  yet 
been  undertaken. 

In  developing  these  instruments,  we  collected  data  on  men  in  ten  Navy  ratings  and 
three  pay  grades  aboard  the  aircraft  carriers  Enterprise  and  Constellation. 

Depending  on  the  job  being  performed  and  the  type  of  instrument,  the  forms  con- 
tain between  34  and  93  rating  items.  A particular  feature  of  this  study  then,  is  the  devel- 
opment of  information  about  performance  in  many  elements  of  a job,  rather  than  with 
regard  to  a few  global  measures. 

The  rating  instruments  are  based  upon  two  different  models  of  job  analysis.  Ernest 
McCormick  (1972)  has  referred  to  these  approaches  as  worker-oriented  and  job-oriented 
models.  A worker-oriented  approach  focuses  on  elements  of  behavior  that  generalize 
across  tasks  and  jobs.  For  example,  observing  visual  displays,  obtaining  information  from 
written  materials,  using  non-precision  tools,  activating  variable  setting  controls,  following 
fixed  procedures,  estimating  quantity,  analyzing  information,  or  negotiating  with  people. 

A job-oriented  approach  to  job  analysis  focuses  on  specific  technological  elements  of 
job  content.  For  example,  repairing  carburetors,  drafting  business  letters,  annealing  copper 
tubing,  organizing  stock  control  functions,  or  translating  Russian  newspaper  articles. 

Forms  of  our  worker-oriented  instrument,  the  PAI,  are  based  upon  elements  of  jobs 
taken  from  McCormick’s  job  analysis  questionnaire,  the  Position  Analysis  Questionnaire 
or  PAQ.  The  jobs  in  the  study  were  first  analyzed  with  a modified  form  of  the  PAQ 
that  we  developed  for  use  in  the  Navy.  Each  job  was  analyzed  by  rating  the  relevance  or 
importance  of  each  of  139  possible  worker-oriented  elements.  Then  performance  rating 
scales  were  developed  for  each  element  of  importance  that  emerged.  (See  Table  1,  page  4) 

This  slide  shows  the  jobs  in  the  study  and  the  number  of  worker-oriented  items 
included  in  the  various  forms  of  the  Performance  Analysis  Inventory  for  obtaining  per- 
formance evaluations. 

Forms  of  our  job-oriented  rating  instrument,  the  Task  Proficiency  Inventory,  were 
based  on  task  inventory  data  furnished  by  the  Navy  Occupational  Task  Analysis  Program 
(NOTAP).  Here,  we  used  existing  task  analysis  data  to  identify  elements  of  performance 
to  be  evaluated.  (See  Overlay  to  Table  1,  page  4) 


'Paper  presented  at  1977  Military  Testing  Association  Conference,  San  Antonio.  Texas,  17-21 
October  1977. 

3 Sponsored  by  the  Naval  Education  and  Training  Command  and  the  Personnel  and  Training 
Research  Program,  Office  of  Naval  Research,  Contract  N00014-75-C-0938  (NR  156-047),  The  project 
monitor  was  Dr.  Marshall  Farr,  Director,  Personnel  and  Training  Research  Program. 
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This  overlay  adds  the  number  of  job-oriented  items  included  in  the  various  forms  of 
the  Task  Proficiency  Inventory  for  obtaining  performance  evaluations.  Here,  separate 
instruments  were  developed  by  pay  grade  where  such  job  analysis  data  was  available.  At 
the  time  we  constructed  the  scales,  NOTAP  job  analysis  data  were  not  available  tor  Met  - 
trician’s  Mate.  Hull  Maintenance  Technician,  and  Interior  Communications.  The  essential 
feature  of  both  methodologies  is  that  performance  is  defined  and  evaluated  in  terms  of 
very  specific  behavioral  or  technological  referents.  (See  Table  '2.  page  5) 

This  slide  shows  some  sample  items  from  the  worker-oriented  PAI  for  Aviation 
Boatswain's  Mate  Equipment.  (See  Table  3.  page  6) 

This  slide  shows  some  sample  items  from  the  job-oriented  TP1  for  the  same  job. 

To  provide  a basis  for  item  analysis,  performance  data  were  obtained  for  5159  incum 
lients  in  the  ten  jobs.  For  comparative  purposes,  we  also  obtained  Performance  hvnlua 
turn  Report  scores.  This  is  the  instrument  that  is  used  operationally  in  the  Navy  for 
evaluating  a man’s  performance.  (See  Figure  la.  page  7) 

This  slide  shows  frequency  distributions  of  scale  value  usage  for  E-3  and  M l on  the 
different  instruments.  The  worker-oriented  PAI  and  the  job-oriented  TIM  have  7 point 
scales,  while  the  Performance  Evaluation  Report  has  ten-point  scales.  The  distributions 
are  displayed  with  the  mid  points  of  the  scales  coinciding  in  order  to  avoid  the  distortion 
that  occurs  if  data  from  one  type  of  scale  are  expressed  in  terms  of  the  other. 

The  PAI  and  TP1  show  less  skew  than  the  operational  instrument,  but  it  must  ho 
(minted  out  that  here  we  are  comparing  experimental  and  operational  data.  Obviously, 
we  do  not  know  what  characteristics  our  instruments  would  demonstrate  if  they  were 
administered  on  a continuing  basis  by  military  personnel.  (See  Figure  lb.  page  81 

This  slide  shows  the  distributions  for  E-5  and  all  pay  grades  combined.  As  always, 
skewness  increases  with  grade.  The  job  element,  or  task  level  approach  lias  not  overcome 
this  problem.  Frequency  data  in  the  handout  shows  respectable  distributions  for  our 
instruments  at  the  E-3  level,  but  deterioration  setting  in  at  the  K-l  level. 

The  handout  also  contains  tables  of  means,  standard  deviations  of  subject  means, 
subject  standard  deviations,  and  item  standard  deviation.  These  analyses  were  undertaken 
to  look  for  relative  leniency,  halo,  and  discrimination  between  the  two  experimental 
rating  instruments.  Comparisons  among  these  statistics  show  less  leniency  and  halo  ami 
better  discrimination  for  the  worker -oriented  scales  than  the  job-oriented  scales.  (Six' 
Tables  4-14,  pages  9-16) 

This  completes  my  presentation.  We  presently  are  awaiting  permission  to  use  these 
instruments  in  the  second  phase  of  our  study  to  collect  data  on  the  performance  of  men 
at  different  aptitude  levels. 

Reference 

Ernest  J.  McCormick,  Paul  R.  Jeanneret,  and  Robert  C.  Medium,  “A  study  of  job 
characteristics  and  job  dimensions  as  based  on  the  position  analysis  questionnaire  (PAQ), 
monograph.  Journal  of  Applied  Psychology,  Vol.  56,  No.  4,  August  1972. 
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Number  of  Items  in  the  Performance  Analysis  Inventory  (PAD 
And  the  Task  Proficiency  Inventory  (TPI)  by  Navy  Job  and  Pay  Grade 


(Overlay  to  Table  1) 


- - , 

NAVY  JOB 

l»AI 

TPI 

IE3 

I 4 

1 

Aviation  Boatswain's  Mate 

Equipment  (ABE) 

bO 

40 

40 

63 

Fuel  (ABF) 

49 

40 

40 

bO 

Handling  (ABM) 

bt> 

40 

40 

bO 

Aviation  Oidnance  (AO) 

45 

b4 

40 

51 

Elect! ician's  Mate  (EM) 

bO 

Flull  Maintenance  Technician  (HI) 

4/ 

Intenoi  Communication  (1C) 

49 

Mess  Management  Specialist 

(MS  S2  Division) 

39 

54 

40 

bt> 

(MS  Sb  Division) 

38 

93 

08 

80 

Storekeeper  (SK) 

34 

bti 

40 

01 
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Table  } 


Sample  Items  From  Worker  Oriented  PAI  for 
Aviation  Boatswain's  Mate  — Equipment 


2.  Work  produced  using  energy  powered  tools  to  perform  operations  not  requiring  great  accuracy 
or  precision.  (Electric  grinders  and  drills,  welding  equipment,  brazing  gear,  skill  saw,  etc  I 


t xceptionaliy 
Good 

Satisfactot  y 

t xceptionally 
Pooi 

7 

6 

b 

4 

3 

2 

1 

3.  Work  accomplished  using  handling  devices.  (Pouring  nnc  from  ladles,  using  mechanical 
fingers,  etc.) 


E weep tiof tally 

Efficient 

Satisfactory 

Exceptionally  Nevet 

Inefficient  Has  to 

; 6 

b 4 

3 

2 

1 X 

Remembering  information  for  a brief  period  of  time, 
timer  readings,  steam  pressures,  etc.) 

(Console  recorder,  launch  valve  strobe 

Very  Meltable 

Satisfactory 

Veiy  Untenable 

7 6 

S 4 

3 

2 

1 

41.  Being  aware  of  and  alert  to  the  condition/quality  of  equipment  material,  or  weapon  systems. 
(For  example,  condition  of  components  in  catapult  and  recovery  gear,  etc.) 


E xcept  tonally 
Aware 

Satistactoi  y 

i xceptionally 
Unawaie 

7 

6 

b 4 

3 

2 

1 

42.  Being  accurate  in  transcribing.  (Copying  or  posting  data  or  information  for  later  use;  water 
brake  readings,  fluid  history  peports,  etc.) 


Exceptionally 

Accurate 

1 

Satisfactory 

Vciy 

Inaccurate 

Nevei 

Has  to 

7 

6 

b 4 

3 

2 

1 

X 

S3.  Obtaining  job  information  by  seeing  differences  using  far  vision.  (Deck  edge  operator,  air 
craft  identification  to  determine  correct  settings  for  arresting  gear,  etc.) 


Exceptionally 

Good 

Satisfactory 

E xcept  tonally 
Poor 

Nevei 

Has  to 

7 

6 

5 

4 

3 

2 

1 

X 

5 


Table  3 


Sample  Items  From  Job-Oriented  TPI  for 
Aviation  Boatswain's  Mate  - Equipment 


Very 

Very 

Never 

Effective 

Average 

Ineffective 

Has  to 

7 6 

5 

4 

3 

2 

1 

X 

I 17.  Applying  preservatives  to  cables  (CDPS,  purchase  cables,  bridles,  etc  ). 
| 18.  Cleaning  hydraulic  biters. 

) 19.  Stowing/breaking  out  parts/equipment 

| 20.  Rigging  the  barricade. 

| 21.  Changing  bridle  arrestor  straps. 

| 22  Replacing  "O"  rings  in  volves/cy Under s. 

| 23  Painting  safety  markings  on  flight  deck. 

| 24  Repacking  the  retract  valve 

| 25.  Maintaining  logs  records  (catapult,  flight  deck,  fuels,  etc  ). 

| 26.  Participating  in  working  parties 
| 27  Functionally  checking  catapults  py  firing  no  loads. 

| 28.  Ensuring  safety  lines  are  in  place  during  no  load  firings. 

| 29  Changing  purchase  cable  on  "AG”  engines  (re  reeve). 

| 30  Safety  wiring  equipment/gear/switches. 

I 31  Measuring  slipper  wear 


'jure  lb  Frequency  Distributions  (in  %)  of  Scale  Value  Usage  for 

Three  Rating  Instruments  Drawn  With  the  Mid  Points  of  Scales  Coinciding 
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Frequency  of  Usage  of  Scale  Values  From  E3  Data  on 
Three  Rating  Instruments  (All  Navy  Jobs,  All  Items) 


T'AI 

TPI 

Pfc  H 

Si 

Nav\ 

Vj  we% 

t 

i 

\ 

1 

1 

■v. 

Scale  Value 

i 

■v. 

. 

t 

430 

4 14 

144 

4 03 

1 0 

1 

16 

2 

686 

6 // 

180 

5 03 

2 0 

7 

1 10 

3 

1306 

1 3 88 

3bb 

9 93 

2 6 

12 

1 88 

4 

28b*i 

38  18 

943 

26  3b 

2 8 

31 

4 8b 

b 

2394 

33  63 

8b  1 

23  78 

3 0 

34 

b 32 

6 

1841 

18  1 7 

786 

21  96 

3 2 

80 

12  b2 

7 

631 

6 33 

330 

8 94 

3 4 

13b 

21  13 

3 6 

174 

27  23 

Nfv  ft 

3 8 

142 

22  22 

Mas  f o 

16  ib 

Ib8b 

4 0 

23 

3 60 

TOT  Al  ’ 

10134 

99  09 

3b  7 9 

100  00 

639 

100  01 

* Totals  ito  not  i net uil*  Npw  Mas  To  pcfomi 
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Frequency  of  Usage  of  Scale  Values  From  E4  Data  on 
Three  Rating  Instruments  (All  Navy  Jobs.  All  Items) 


Values 

PAI 

: 

TPI 

m* 

i 

•v 

i 

■v 

Navv 

Scale  Value 

t 

■v 

1 

9b 

0 92 

37 

1 00 

1 0 

2 

241 

2 32 

50 

1 35 

2 0 

3 

999 

9 63 

209 

b 63 

26 

4 

46 

4 

2349 

22  6b 

762 

20  53 

28 

18 

2 09 

5 

2821 

27  20 

909 

24  50 

3 0 

30 

3 48 

6 

2758 

26  60 

1122 

30  23 

3 2 

47 

b 4b 

7 

1 107 

10  68 

622 

16/6 

3 4 

123 

14  27 

3 6 

292 

33  88 

Never 

3.8 

279 

32  37 

Has  To 

683 

1444 

4 0 

69 

801 

TOT  Al * 

103/0 

100  (XI 

3711 

11X100 

862 

100  01 

Totals  tlo  not  •ncluile  Has  To  p«nlo»m 


Table  6 


Frequency  of  Usage  of  Scale  Values  From  E5  Data  on 
Three  Rating  Instruments  (All  Navy  Jobs,  All  Items) 


PAI 

TPl 

PER 

Scale 

Values 

f 

% 

f 

% 

Navy 

Scale  Value 

f 

% 

i 

4 

0.13 

2 

0.13 

1.0 

2 

14 

0.46 

16 

1.07 

2.0 

3 

98 

3.18 

13 

0.87 

2.6 

4 

355 

11.53 

119 

7.96 

2.8 

5 

915 

29.71 

326 

21.79 

3.0 

1 

.98 

6 

1052 

34.16 

571 

38.17 

3.2 

1 

.98 

7 

642 

20.84 

449 

30.01 

3 4 

4 

3.92 

Never 

3.6 

29 

28.43 

3.8 

47 

46.08 

Has  To 

118 

459 

4.0 

20 

19.61 

TOTAL" 

3080 

100.01 

1496 

99.99 

102 

100.00 

•Totals  do  not  include  "Never  Has  To"  perform. 
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Frequency  of  Usage  of  Scale  Values  From  E3-E5  Data  on 
Three  Rating  Instruments  (All  Navy  Jobs,  All  Items) 


PAI 

1 

TPl  1 

PER 

Scale 

Navy 

Values 

f 

% 

f 

% 

Scale  Value 

f 

% 

i 

519 

2.20 

183 

2.08 

1 0 

1 

.06 

2 

941 

3.99 

246 

2.80 

2 0 

7 

44 

3 

2403 

10.19 

577 

6.57 

2.6 

16 

1.00 

4 

5560 

23.58 

1824 

20.76 

2.8 

49 

3.06 

5 

6130 

25.99 

2086 

23.74 

3.0 

65 

4.06 

6 

5651 

23.96 

2479 

28.22 

3.2 

128 

7.99 

7 

2380 

10.09 

1391 

15.83 

3.4 

262 

16.34 

Never 

Has  To 

2416 

3488 

3.6 

3.8 

495 

468 

30.88 

29.20 

4.0 

112 

6.99 

TOTAL’ 

’ 23584 

100.00 

8786 

100.00 

1603 

100.02 

•Totals  do  not  include  "Never  Has  To''  perform. 
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Moans  and  Standout  Deviations  ot  Subject  Means  on 
Three  Hating  Instruments 


Mean 

StAmlAuf  IVviomm 

rottwt  iihi 

TAI 

n'i 

p»  n 

I’AI 

tpi 

PI  M 

N 

Hum 

I’AI  A 

N 

PI  M 

m 

1*1  M 

Mmh 

AIU 

4 44 

4 71 

S OS 

1 29 

1 26 

04 

46 

20 

3 42 

AW 

4 13 

4 36 

6 36 

1 23 

1 13 

68 

23 

13 

3 61 

ABM 

4 64 

4 i»7 

6 2? 

1 18 

1 04 

1 06 

42 

33 

3 4 7 

AO 

4 30 

4 77 

6 28 

1 52 

1 37 

83 

37 

24 

3 40 

1 M 

4 36 

4 1)4 

1 (X) 

1 19 

17 

13 

3 36 

HT 

4 29 

6 78 

1 .>8 

66 

ID 

•) 

3 63 

1C 

4 73 

5 30 

76 

33 

8 

6 

3 52 

mss; 

3 47 

3 82 

4 86 

1 22 

1 21) 

1 20 

20 

12 

3 36 

MS  S'j 

4 29 

4 66 

6 38 

1 33 

1 34 

00 

33 

17 

3 62 

SK 

4 Hi 

4 63 

4 77 

1 23 

1 03 

88 

10 

6 

3 30 

1 4 


AH1 

6 1!) 

6 26 

6 6 1 

8!) 

92 

73 

22 

12 

3 66 

All) 

5 00 

6 14 

6 67 

8 7 

02 

64 

28 

18 

3 IX) 

ABH 

6 31 

6 3 1 

6 01 

1 12 

1 12 

46 

30 

24 

3 70 

AO 

4 8 7 

6 02 

6 61) 

1 20 

1 27 

63 

36 

28 

3 61 

1 M 

4 40 

6 26 

1 10 

6/ 

31 

30 

3 48 

111 

4 76 

6 63 

88 

1)4 

24 

22 

3 60 

1C 

4 45 

6 30 

1)1) 

63 

16 

13 

3 41) 

MS  S2 

6 01 

6 12 

6 73 

1 IX) 

) 03 

64 

23 

16 

3 62 

MS  S6 

6 61 

6 IK) 

6 41) 

66 

64 

60 

12 

8 

3 86 

SK 

6 12 

6 66 

6 0/ 

87 

72 

63 

18 

14 

3 60 

I !i 


AIM 

6 46 

6 68 

6 60 

<3(1 

87 

NC- 

0 

J 

3 89 

Alii 

6 6 1 

6 74 

6 87 

63 

60 

1 7 

8 

3 66 

Aim 

6 78 

6 86 

• • 

NC 

NC 

• • 

3 

* * 

• • 

AO 

6 20 

6 14 

• • 

1 IX) 

1 J7 

• • 

6 

• • 

• * 

1 M 

6 03 

6 07 

26 

68 

10 

4 

3 72 

HT 

NC 

NC 

NC 

NC 

■j 

2 

NC 

1C 

467 

NC 

NC 

NC 

3 

NC 

MS  S2 

6 36 

6 64 

NC 

86 

86 

NC 

1 1 

1 

NC 

MS  S6 

6 IX) 

6 26 

■ • 

87 

IX) 

• 1 

1/ 

• • 

# • 

SK 

5 32 

6 73 

6 23 

1 111 

1 IX) 

61 

7 

4 

3 7 7 

I .1  t !i 


All! 

4 77 

4 06 

6 28 

1 22 

1 16 

08 

77 

46 

3 49 

ABF 

4 74 

4 01 

6 6/ 

1 12 

1 08 

64 

50 

36 

3 57 

ADM 

4 96 

6 14 

6 66 

1 111 

1 07 

04 

76 

6 7 

3 67 

AO 

4 60 

4 07 

6 61 

1 45 

1 34 

76 

78 

62 

3 66 

IM 

4 4!) 

6 22 

1 1X1 

87 

68 

48 

3 47 

HT 

4 62 

5 72 

1 06 

86 

46 

34 

3 62 

1C 

4 66 

6 35 

87 

66 

26 

22 

3 61 

MS  S2 

4 61 

4 72 

6 36 

1 36 

1 .10 

1)8 

64 

20 

3 51 

MS  S5 

6 IX) 

6 28 

6 74 

I 34 

1 32 

06 

62 

26 

3 62 

SK 

4 89 

6 38 

6 78 

1 12 

00 

81 

36 

23 

3 63 

OVtRAl 1 

4 73 

6 04 

6 4!) 

1 23 

1 20 

86 

1‘AI  660 

370 

I I'l  440 
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T i»l*le  9 


Number  of  Times  Means  on 
One  Rating  Instrument  Exceed  Another 


PAI  > TPI  PA  I PE  H TPI  • PER 

E3  0/7  0/10  0/7 

E4  0/6"  0/10  0/7 

E5  1/7  0/7"  0/4" 

“A  tie  occurred  in  one  comparison. 

**PER  data  were  not  available  for  3 Navy  jobs,  thus  rtHiucmii  the 
number  of  comparisons  that  could  he  made  between  I*  A I and  PFR  to  7, 
and  tie  tween  TPI  and  PER  to  4 


Table  10 

Number  of  Times  Standard  Deviations  of 
Means  on  One  Rating  Instrument  Exceed  Another 


PAI  > TPI 

PAI  > PEK 

1 PI  > PE  11 

E3 

5/7 

9/10 

6/7 

E4 

3/6 

8/10 

7/7 

E5 

4/6 

2/3 

2/2 

Table  1 1 


Mean  of  Subject  Standard  Deviations  for 
Three  Rating  Instruments 


E3 

E4 

E5 

E3-E5 

PAI 

ABE 

084 

094 

059 

084 

ABF 

0 72 

0.67 

069 

0.69 

ABH 

086 

0.74 

0.63 

0 80 

AO 

0.65 

0.67 

0.73 

0.67 

EM 

0.75 

0.79 

0.68 

0.76 

HT 

0.55 

0.64 

NC* 

0.60 

1C 

0.71 

0.71 

0.75 

0 71 

MS-S2 

0.70 

0.71 

0.62 

0.69 

MSS5 

0.69 

0.68 

060 

0 66 

SK 

0 76 

0.73 

0.66 

0.73 

TPI 

ABE 

0.80 

083 

0.77 

081 

ABF 

0.76 

0.66 

0.65 

0 70 

ABH 

0.74 

0.78 

0.69 

0.75 

AO 

0.61 

0.68 

0.62 

0 64 

EM 

* e 

■ e 

• • 

• • 

HT 

» • 

e * 

• • 

* • 

1C 

* i 

e • 

e • 

e • 

MS-S2 

0.62 

0.71 

0.56 

0.65 

MS-S5 

0.57 

0.61 

0.56 

0.57 

SK 

0.51 

0.67 

0.66 

0.62 

per 

ABE 

0.47 

0.66 

0.29 

0.51 

ABF 

0.67 

048 

0.31 

0.54 

ABH 

0.60 

0.47 

NC 

NC 

AO 

0.50 

0.54 

NC 

NC 

EM 

0.47 

060 

066 

0.57 

HT 

0.36 

0.44 

NC 

NC 

1C 

0.56 

0.52 

NC 

NC 

MS-S2 

0.43 

0.47 

NC 

NC 

MS  S5 

0.33 

0.33 

NC 

NC 

SK 

0.47 

0.35 

0.46 

040 

*The  number  of  cases  was  so  low,  that  the  statistic  was  not  computed  where  NC  is  shown. 
**No  data  were  available. 
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I I Table  12 

Number  of  Times  Average  Subject  Standard  Deviations  on 
One  Rating  Instrument  Exceed  Another  for  a 
Given  Pay  Grade 


PAI  > TPI 

PAI  > PER 

TPI  > PER 

E3 

6/7 

10/10 

7/7 

E4 

4/6 

10/10 

7/7 

E5 

5/7 

4/4 

3/3 

\ 

I 

\ 
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Mean  of  Item  Standard  Deviations  for  Three  Rating  Instruments 


E3 

E4 

E5 

E3E5 

*so 

^Sl) 

XSD 

*SD 

PAI 

ABE 

1 52 

1 18 

1.14 

1 42 

ABF 

1 38 

1 09 

0 88 

1 26 

ABH 

1 40 

1 27 

0 62 

1 36 

AO 

1 60 

1 42 

1.17 

1 56 

EM 

1.26 

1.35 

0 74 

1 21 

HT 

1 42 

1 12 

0.16 

1 24 

1C 

1 03 

1.15 

0 86 

1 10 

MS  S2 

t 36 

1.31 

1 00 

1 49 

MS  S5 

1.39 

0.85 

098 

1.31 

SK 

1 34 

1 07 

1.23 

1 27 

TPI 

ABE 

1 45 

1 19 

1 07 

1 28 

ABF 

1 26 

1 15 

081 

1 24 

ABH 

1.26 

1 27 

0 49 

1 09 

AO 

1 52 

1 44 

0 97 

1 18 

EM 

• * 

• • 

• * 

• « 

HT 

• * 

• * 

• • 

• • 

1C 

* • 

• • 

• • 

• • 

MS  S2 

1 52 

1.22 

0 96 

1 45 

MS  S5 

1.52 

080 

0 84 

1.21 

SK 

1 05 

100 

1 05 

0 98 

PER 

ABE 

1 05 

088 

0 38 

1 03 

ABF 

0.72 

0.78 

0 40 

0 81 

ABH 

1 07 

0.60 

NC* 

098 

AO 

1 05 

082 

NC 

0 94 

EM 

1 36 

0 88 

0 75 

1 05 

HT 

0.61 

1 02 

0 28 

097 

1C 

055 

081 

0 56 

0 75 

MS  S2 

1 26 

0 80 

NC 

1.03 

MS  S5 

0 98 

0 61 

NC 

092 

SK 

1 03 

0 70 

0 78 

092 

Thi*  number  of  cases  was  so  low,  that  the  statistic  was  not  computed  where  NC  is  shown. 
No  data  were  available 
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Table  14 


Number  of  Times  Average  Item  Standard  Deviations  on 
One  Rating  Instrument  Exceed  Another  for  a 
Given  Pay  Grade 


PAI  > TPI 

PAI  > PER 

TPI  ^ PER 

E3 

7/9 

9/10 

7/7 

E4 

3/C 

10/10 

7/7 

E5 

7/7 

5/7 

4-4 

it 


